On the Power of Democratic Networks

Linear Threshold Boolean units LTUs are the basic processing com ponents of arti cial neural networks of Boolean activations Quantization of their parameters is a central question in hardware implementation when numerical tech nologies are used to store the con guration of the circuit In the previous studies on the circuit complexity of feedforward neural networks no di erences had been made between a network with small integer weights and one composed of majority units LTUs with weights in f g since any connection of weight w w integer can be simulated by jwj connections of value sgn w This paper will focus on the circuit complexity of democratic networks i e circuits of majority units with at most one connection between each pair of units The main results presented are the following any Boolean function can be com puted by a depth non degenerate democratic network and can be expressed as a linear threshold function of majorities AT LEAST k and AT MOST k are com putable by a depth polynomial size democratic network the smallest sizes of depth circuits computing PARITY are identical for a democratic network and for a usual network the VC dimension of the class of the majority functions is n i e equal to that of the class of any linear threshold functions Acknowledgements The author gratefully acknowledge the Swiss National Science Foundation grant